An Evaluation Methodology for Parallel/Distributed Software Tools

The recent rapid growth of the network computing applications area has been accelerated by a variety of parallel and distributed computing (PDC) software tools that simplify process management, inter-process communication, and program debugging in a PDC environment. This variety of software tools varies signi cantly in terms of the application domain targeted and corresponding functionality provided, the computational and communication model supported, the underlying implementation philosophy, and the computing environments supported. This makes the selection of the best tool to run a given class of applications on a parallel or distributed system a non-trivial task that requires some investigation. Currently, there are no general criteria to evaluate PDC software tools, nor is it easy to lay down such criteria. In this paper we present a multilevel evaluation methodology for PDC software tools in which tools are evaluated from three di erent perspectives: tool performance level (TPL), application performance level (APL), and application development level (ADL). In TPL the performance of the basic communication primitives provided by the tools (e.g., pointto-point communication and collective communication primitives) is evaluated. The APL evaluates the performance of representative applications developed using di erent tools and the ADL evaluates the tool capability to support and facilitate the development of parallel and distributed applications. The main objective of the research presented in this paper is to provide a means for evaluating, quantifying, and comparing the suitability of PDC software tools in terms of user requirements and to help PDC software tool developers to de ne the requirements of future tools by identifying the de ciencies and bottlenecks in existing systems. We apply our evaluation methodology to four PDC message-passing tools: i.e., p4, PVM, MPI, and NCS. These tools are evaluated over a cluster of workstations interconnected by an ATM network.